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EXECUTIVE  SUMMARY 


The  Operations  Research  Center  (ORCEN),  a  United  States  Military  Academy 
Center  of  Excellence,  conducted  an  analysis  of  the  Intern  Forecasting  System  (IFS).  This 
was  at  the  request  of  the  Director,  Civilian  Personnel  Management  Directorate,  of  the  Total 
Army  Personnel  Command.  The  General  Research  Corporation  (GRC)  is  the  agency 
contracted  to  develop  the  IFS,  which  is  a  component  of  the  Civilian  Forecasting  System 
(CIVFORS). 

The  ORCEN's  mission  was  to  study  the  IFS  to  determine  how  it  produces  forecasts 
of  intern  personnel,  and  to  determine  some  reasons  why  the  IFS  has  not  produced 
seemingly  reasonable  results  to  date. 

The  results  of  this  study  are  that  GRC  has  developed  a  technically  adequate  solution 
to  the  problem  they  faced,  but  they  have  not  designed  an  appropriate  solution  that  performs 
well  in  the  user's  environment  and  assures  user  satisfaction. 

After  several  operational  tests  the  IFS  has  not  yet  provided  reasonable  results.  In 
fact  for  numerous  career  program  and  major  command  categories  the  IFS  forecasted  zero 
requirements  across  the  entire  Five  Year  Defense  Plan.  This  is  due  in  part  to  ineffective 
settings  for  conditional  parameters  that  control  which  forecasting  model  to  use,  and 
seasonality  parameters  that  enable  the  principle  forecasting  model,  Winter's  model,  to 
forecast  more  accurately. 

The  output  screens  require  the  users  to  manually  aggregate  hundreds  of  data  to 
achieve  the  results  they  need.  This  manual  aggregation  could  be  handled  very  effectively 
within  the  IFS.  Additionally,  the  menuing  systems  requires  that  the  user  access  multiple 


levels  of  information  before  they  can  begin  to  aggregate  this  data  manually.  It  is  apparent 
that  the  user's  needs  and  operating  environment  were  given  small  consideration  in  the 
development  of  the  IFS  to  date. 

The  Army  Civilian  Training  and  Education  Systems  (ACTEDS)  branch  of  the 
Civilian  Personnel  Management  Directorate  is  justified  in  being  hesitant  to  accept  the  IFS 
in  its  current  form.  The  IFS  has  not  provided  reliable  results,  and  the  results  that  it  has 
provided  require  literally  days  of  manual  calculations  to  determine  the  needed  information. 
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1.  OVERVIEW 


a.  Background 

The  Intern  Forecasting  System  (IFS)  is  a  component  of  a  larger  government  contract 
known  as  the  Civilian  Forecasting  System  (CIVFORS),  the  civilian  component  to  the 
Headquarters,  Department  of  the  Army  Decision  Support  System  (HQDA  DSS).  The 
contract  to  develop  and  field  CIVFORS,  including  the  IFS,  was  awarded  to  the  General 
Research  Corporation  (GRC). 

The  IFS  was  initiated  in  May  1989,  when  the  revised  Statement  of  Work  was  signed 
between  GRC  and  the  Army  Civilian  Training  and  Education  Development  System 
(ACTEDS)  Branch,  Directorate  of  Civilian  Personnel  Management,  Total  Army  Personnel 
Command  (TAPC).  The  IFS  is  intended  to  provide  systematic  forecasts  of  future  intern 
needs. 


In  May  1991  an  operational  test  of  the  IFS  was  conducted  by  GRC.  The  results  of 
this  test  provided  some  incentive  for  ACTEDS  branch  personnel  to  seek  outside  technical 
assistance  to  determine  how  the  IFS  operates,  and  if  its  output  is  valid. 

The  Civilian  Personnel  Management  Office,  TAPC  asked  the  Operations  Research 
Center  (ORCEN),  a  Center  of  Excellence  at  the  United  States  Military  Academy  (USMA),  to 
conduct  this  technical  analysis.  ORCEN  personnel  made  two  trips  to  TAPC.  The  first  trip 
was  planned  to  become  familiar  with  the  key  personnel  involved  with  the  IFS  from  both 
ACTEDS  branch  and  GRC,  as  well  as  the  IFS  model  itself.  During  the  second  trip,  ORCEN 
personnel  conducted  a  thorough,  detailed  analysis  of  the  IFS  operating  method  with  all  key 
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personnel  from  both  ACTEDS  branch  and  GRC  in  attendance.  This  report  is  the  result  of 
this  work  and  some  continued  analysis  conducted  at  USMA. 

b.  Purpose 

This  report  has  two  purposes.  First,  to  provide  ACTEDS  branch  personnel  with  the 
results  of  the  detailed  analysis  that  the  ORCEN  conducted.  Second,  to  provide  ACTEDS 
branch  personnel  a  description  of  the  IFS  operational  method  in  non-computer  terms.  The 
second  purpose  is  provided  in  Appendix  A.  It  is  the  hope  of  the  ORCEN  that  this  report  can 
be  used  as  a  stepping  stone  toward  a  cooperative  and  useful  completion  of  the  IFS 
development. 


2.  EVALUATION 

a.  Methodology 

This  effort  focuses  on  four  issues  related  to  designing  successful  systems  which  are 
the  subdivisions  of  this  report  (Rouse,  1991).  They  are  as  follows: 

Has  the  right  problem  been  formulated? 

Has  an  appropriate  solution  been  designed? 

Has  the  IFS  been  developed  to  perform  well? 

Does  the  IFS  assure  user  satisfaction? 


These  issues  reflect  the  "users"  point  of  view  which  is  the  perspective  held 
throughout  this  report.  Since  the  users  of  the  IFS  are  people  assigned  to  ACTEDS  branch, 
the  structure  of  the  discussion  focuses  on  human-centered  design. 

Human-centered  design  has  recently  gained  increased  attention  in  the  world  of 
systems  analysis.  Dr.  William  B.  Rouse  describes  a  framework  to  study  a  system  based 
upon  human-centered  design  in  his  book,  Design  for  Success  -  A  Human-Centered  Approach 
tp  Designing  Successful  Products  and  Systems.  He  describes  the  tenets  of  human-centered 
design  as  being  focused  on  three  objectives.  First,  a  system  should  be  designed  to  enhance 
human  abilities,  adding  speed  to  operations  humans  would  do  manually.  Second,  a  system 
should  be  designed  to  overcome  human  limitations,  providing  efficient  means  to  overcome 
common  human  errors.  When  one  designs  a  system  that  is  to  be  used  by  people,  these  first 
two  objectives  not  only  make  common  sense,  they  are  essential.  Following  naturally  from 
the  first  two  objectives,  the  third  objective  is  to  foster  user  acceptance.  Understanding  the 
user's  concerns  and  preferences  is  not  enough.  Implementing  a  design  that  satisfies  those 
concerns  and  preferences  in  a  highly  usable,  "friendly"  format  is  the  important  point  of 
human-centered  design. 

b.  Results 

This  report  focuses  on  each  question  separately,  providing  analysis  and  conclusions 
based  upon  the  objectives  of  human-centered  design.  Additionally,  each  question  is 
answered  with  respect  to  the  database  management  and  the  forecasting  components  of  the 
IFS. 


(1)  Has  the  right  problem  been  formulated? 
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GRC  has  developed  a  model  that  does  provide  a  solution  to  the  problem  of 
systematically  determining  a  forecast  for  civilian  intern  requirements  for  the  budget  year 
and  the  Five  Year  Defense  Plan  (FYDP).  Included  in  the  problem  is  reading  the  current 
personnel  strengths  for  interns  in  all  career  programs  (CP),  Major  Commands  (MACOM), 
and  paygrades  (PG),  from  the  HQ  DSS  database,  which  is  updated  on  a  monthly  basis.  With 
respect  to  the  database  management,  GRC  understood  the  problems  associated  with 
obtaining  the  raw  data  and  manipulating  it  into  a  form  which  could  be  easily  accessed  and 
used  by  the  model.  They  went  to  great  lengths  to  identify  and  use  all  data  which  could  be 
applicable  to  the  model.  The  database  module  is  followed  by  the  IFS  forecasting  modules 
which  provide  forecasting  rates  from  the  given  data. 

The  functional  requirements  listed  in  the  Statement  of  Work  indicate  that  the 
problem  to  be  solved  was  understood  at  first,  at  least  in  a  broad  perspective.  Appendix  B 
describes  how  the  IFS  does  not  include  many  of  the  requirements  listed  in  the  Statement  of 
Work. 

(2)  Has  an  appropriate  solution  been  designed? 

As  previously  stated,  the  IFS  does  provide  a  forecast  of  Intern  personnel 
requirements.  However,  the  scope  of  the  solution  which  the  IFS  provides  leaves  much  of  the 
problem  unsolved.  Specifically,  because  the  IFS  focuses  its  forecasts  on  a  level  that  is 
detailed  down  to  the  career  program,  MACOM  and  paygrade,  this  data  must  still  be 
aggregated  to  meet  the  needs  of  ACTEDS  branch.  What  ACTEDS  branch  needs  are 
personnel  forecasts  that  are  aggregated  across  all  paygrades  of  interns,  yet  retain  the  detail 
by  career  program  and  MACOM.  GRC  claims  since  this  is  not  specified  in  the  Statement  of 
Work,  then  they  are  not  obligated  to  provide  the  data  in  this  form.  While  this  may  be 
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legally  true,  it  also  seems  a  waste  to  provide  a  forecasting  model  that  produces  output  that 
requires  hundreds  of  simple  additions  and  subtractions  to  compute  the  desired  forecasts. 


The  solution  developed  by  GRC  to  obtain  the  data  and  transform  it  into  an  appropriate 
format  is  very  good.  GRC  has  been  faced  with  the  task  of  obtaining  transaction  data  which 
may  not  always  contain  accurate  fields,  usually  caused  by  data  entry  errors,  or  which  may 
be  several  months  old.  GRC  has  designed  what  appears  to  be  a  very  thorough  and  well 
thought  out  solution  to  the  likely  problems  they  would  encounter  with  the  data  they 
received. 

The  IFS  uses  two  models  to  make  forecasts.  If  conditions  using  two  parameters, 
'‘Small  Cell  Size"  and  "Sufficient  History",  are  met,  then  the  IFS  uses  Winter's 
Approximation  Model  (Winter's)  to  make  the  forecast.  If  either  of  the  conditions  are  not 
met,  then  the  IFS  uses  a  simple  Weighted  Moving  Average  (WMA)  model.  While  both 
methods  are  valid  forecasting  techniques,  neither  is  necessarily  the  best  model  for  this 
situation. 

Different  forecasting  models  are  suited  to  different  trends  in  the  past  data. 
Specifically,  the  WMA  model  relies  on  the  stability  of  past  data.  It  is  best  used  when  the 
data  vary  randomly  around  one  stable  point.  On  the  other  hand,  Winter's  model  can  be 
used  on  data  that  have  had  a  trend  upwards  or  downwards,  and  have  the  added  facet  of 
seasonality.  Seasonality  is  the  natural  cycle  of  increases  followed  by  decreases  that  some 
data  sets  demonstrate.  A  cycle  begins  as  the  data  increase,  peak  out,  go  through  a  decrease, 
stabilize,  and  stops  as  the  data  begin  to  increase  again  into  the  next  cycle.  The  number  of 
cycles  within  a  prediction  interval  determines  the  number  of  "seasons"  to  include  in  the 
model  parameters. 
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All  mathematical  forecasting  models  determine  future  values  based  upon  the  way 
the  data  behave  in  the  past.  Consequently,  relying  on  the  forecasts  of  these  models  is  not 
unlike  predicting  where  a  train  is  heading,  while  looking  out  the  back  of  the  caboose.  The 
point  is  that  forecasting  models  rely  heavily  upon  past  data. 

Thus,  to  forecast  one  must  understand  what  trends  the  data  have  exhibited  in  the 
past  to  decide  which  model  to  use.  Unfortunately,  GRC  has  never  conducted  a  plot  of  the 
data  for  any  category  of  career  program,  MACOM,  and  paygrade  (Bartlett,  1992).  Their 
rationale  for  not  doing  so  has  been  that  although  Winter's  model  may  not  be  the  most 
appropriate  model  to  forecast  intern  personnel  strengths,  it  is  the  most  flexible  model, 
covering  any  trend  and  seasonality  the  data  may  exhibit  (Moore,  1992).  This  rationale 
demonstrates  that  GRC  has  fit  a  model  to  a  problem,  but  has  not  focused  sufficiently  on  the 
problem  they  have  been  contracted  to  solve.  It  is  the  characteristics  of  the  data  itself  that 
should  determine  which  model  to  use. 

(3)  Has  the  IFS  been  developed  to  perform  well? 

Simply,  the  IFS  has  not  performed  well  yet.  On  the  first  test  run  in  May  1991,  a 
number  of  CP/MACOM/PG  categories  had  all  zeroes  forecasted  for  the  entire  FYDP.  This  is 
a  highly  unlikely  event,  and  even  more  unlikely  across  many  categories.  Again,  the  reason 
for  the  unusual  zero  forecasts  is  because  several  of  the  IFS  parameters  are  not  adjusted 
properly.  Consequently,  even  with  the  flexibility  of  Winter’s  model,  the  IFS  typically  does 
not  use  it.  During  the  analysis  of  the  IFS,  several  parameters  and  forecasting  sequences 
were  tested.  As  mentioned  before,  these  parameters  control  which  forecasting  model  to 
employ,  Winter’s  or  WMA.  The  parameters  were  found  to  be  too  strict,  causing  most  of  the 
forecasts  to  be  generated  from  the  simple  Weighted  Moving  Average  model  (White  Paper, 
1991).  Additionally,  because  the  scope  of  the  forecasts  is  so  detailed,  the  overall  results  are 
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often  skewed  downward.  Again,  the  scope  should  be  broader,  to  provide  not  only  better 
forecasts,  but  also,  to  satisfy  better  the  needs  of  the  user,  ACTEDS  branch. 

One  other  note  should  be  made  about  Winter’s  model.  Its  reliability  depends  heavily 
upon  the  number  of  seasons  in  a  prediction  period.  The  forecast  of  interns  is  analyzed  on  a 
yearly  basis.  Thus,  to  use  Winter's  model  properly,  one  would  have  to  know  the  number  of 
seasons  there  are  in  the  data  in  a  year.  GRC  has  assumed  that  the  number  of  seasons  is  4, 
but  they  have  not  plotted  the  data  or  even  asked  ACTEDS  branch  personnel  to  verify  this 
(Bartlett,  1992).  In  discussions  with  ACTEDS  branch  personnel  there  is  no  one  seasonality 
that  could  be  generalized  across  all  career  programs,  MACOMs,  and  paygrades.  Many 
career  programs  tend  to  have  an  increase  in  June,  after  high  school  and  college  graduations, 
and  another  increase  in  the  fall  months  around  September  (Coutcher,  1992).  However,  this 
indicates  that  the  past  data  generally  have  had  2  seasons  per  year.  Unfortunately,  GRC  has 
arbitrarily  assumed  4  seasons  per  year.  The  overall  reliability  of  the  forecasts  generated  by 
the  IFS  can  be  improved  by  more  accurately  knowing  the  number  of  seasons  for  each  data 
set. 


GRC  was  asked  if  they  conducted  any  performance  testing  or  validation  of  the  IFS 
against  actual  intern  needs  between  the  last  forecast  (May  1991)  and  January  1992.  In 
other  words,  they  were  asked  if  they  validated  the  forecast  the  IFS  provided  for  the  months 
between  May  1991  and  January  1992  against  the  actual  requirements  experienced.  Their 
response  was  no.  Further,  they  said  they  would  not  do  such  a  test  without  a  Request  for 
Engineering  Support  (RES)  (Bartlett,  1992).  This  response  seems  to  indicate  that  GRC  may 
realize  that  the  IFS  has  not  performed  well.  Additionally,  it  may  indicate  GRC’s  belief  that 
although  they  may  have  provided  a  technically  adequate  solution,  they  may  not  have 
developed  it  to  assure  user  satisfaction. 
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Once  the  models  have  provided  the  forecasting  rates,  the  next  step  is  to  analyze  the 
sequence  in  which  the  rates  are  applied  to  the  data.  For  example,  for  any  CP/MACOM/PG 
category  there  are  loss  and  gain  rates  that  are  provided  by  the  model  based  upon  the  data 
history.  These  rates  are  multiplied  with  the  current  population  strength  to  determine  the 
loss  or  gain  to  the  population  strength  of  the  future.  This  product  is  then  subtracted  or 
added  to  the  current  population.  Then,  the  next  transaction  rate  is  multiplied  with  this 
updated  version  of  the  current  population  strength,  and  so  on.  The  trouble  with  this 
procedure  is  that  the  sequence  of  transaction  rates  effect  the  population  strengths 
significantly.  In  other  words,  another  application  sequence  of  loss  and  gain  rates  would 
produce  a  uniquely  different  forecast.  When  questioned  as  to  the  logic  of  the  sequence  that 
the  IFS  imposes,  GRC  could  not  remember  their  logic  for  choosing  that  particular  order 
(Moore,  1992).  They  did,  however,  offer  to  provide  ACTEDS  branch  with  a  memorandum 
describing  the  sequence  of  transaction  rates  as  they  are  in  the  IFS  now. 

Finally,  the  IFS  program  does  take  a  long  time  to  run.  This  is  partly  due  to  the 
processing  of  the  incoming  data  to  check  for  validity  and  to  place  it  into  the  format  used  by 
the  IFS  forecasting  models.  This,  however,  appears  to  be  necessary  due  to  the  format  and 
error  rate  of  the  data  provided  by  the  HQ  DSS.  Also,  the  code  written  by  GRC  personnel  is 
well  documented,  making  it  easy  to  follow  and  maintain. 

(4)  Does  the  IFS  assure  user  satisfaction? 

Given  that  the  IFS  has  not  yet  provided  reasonable  forecasts,  it  can  be  concluded 
fairly  that  it  does  not  assure  user  satisfaction.  However,  user  satisfaction  goes  beyond 
providing  just  a  forecast  of  data. 
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GRC  has  done  a  good  job  of  managing  the  database  information.  They  appear  to 
have  taken  great  care  in  performing  these  functions,  possibly  at  the  expense  of  some 
computer  efficiency.  However,  as  far  as  the  user  is  concerned,  the  database  portion  of  the 
model  is  transparent. 

What  is  visible  to  the  user  is  a  menu  system  between  the  IFS  and  the  user,  which  is 
not  very  "friendly"  or  state-of-the-art.  The  menus  are  cumbersome  to  use,  requiring  the 
user  to  negotiate  numerous  levels  of  information  to  obtain  the  forecasts.  This  portion  of  the 
model  should  have  received  more  thought  and  effort.  GRC's  response  to  this  is  that  they 
were  limited  to  using  Cross  System  Product,  a  menuing  language  available  on  the  HQ  DSS 
computer.  This  could  be  used  to  explain  most  of  the  clumsiness.  However,  more  thought 
and  foresight  still  could  have  been  placed  into  the  menuing  system  to  make  the  IFS  more 
"user-friendly”.  A  simple  prototype  approach,  where  users  review  draft  screens,  would  have 
alleviated  this  problem. 

As  alluded  to  previously,  the  forecasts  are  displayed  in  a  manner  that  is  too  detailed. 
ACTEDS  branch  personnel  must  aggregate  the  forecasted  data,  by  hand,  to  conduct  analysis 
of  intern  planning  and  costs.  It  would  be  a  small  matter  to  aggregate  the  forecasted  data  in 
the  IFS  program  itself,  alleviating  the  need  for  ACTEDS  branch  personnel  to  aggregate  it 
manually.  Unfortunately,  GRC  has  insisted  that  to  modify  the  way  the  IFS  displays  the 
output  would  require  an  Engineering  Change  Proposal  (ECP),  which  will  require  more 
funding  (Bartlett,  1992).  This  is  just  another  example  of  how  GRC  can  do  a  better  job  of 
designing  a  system  to  ensure  user  satisfaction. 

Even  with  the  questionable  results  provided  by  the  first  tests  of  the  IFS,  GRC 
maintains  that  they  have  fulfilled  the  requirements  of  the  Statement  of  Work 
(Bartlett, 1992).  Further,  ACTEDS  branch  personnel  would  have  to  accept  the  IFS  as  is, 
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before  any  ECPs  to  modify  the  IFS  model  could  be  accepted  to  bring  the  IFS  forecasts  into  a 
more  usable  format. 

Another  important  part  of  assuring  user  satisfaction  for  a  computer-based  system  is 
an  easy  to  read  and  understand  user's  manual.  The  documentation  that  GRC  provides  with 
the  IFS  is  very  inadequate.  It  is  difficult  to  read  and  hard  to  understand. 


3.  CONCLUSION 

The  General  Research  Corporation  has  provided  a  product  to  ACTEDS  branch  which 
they  claim  fulfills  their  contract  obligations  as  outlined  in  the  Statement  of  Work 
agreement.  Located  in  Appendix  B  are  many  instances  of  items  listed  in  the  Statement  of 
Work  that  are  not  addressed  by  the  IFS  in  its  current  state.  These  items  of  difference  have 
been  brought  to  GRC's  attention  previously.  They  have  responded  with  a  paper  explaining 
why  certain  items  have  not  been  included.  However,  the  question  remains  that  if  they 
agreed  to  a  Statement  of  Work,  then  should  not  all  items  listed  on  it  be  satisfied? 

This  report  illuminates  several  issues  about  the  IFS  model  which  can  be  used  to 
improve  its  reliability  and  usefulness.  If  GRC  adjusts  the  condition  parameters  of  the  IFS 
that  dictate  which  model,  Winter's  or  Weighted  Moving  Average,  as  they  stated  they  would 
do  during  the  second  ORCEN  meeting,  then  perhaps  the  IFS  may  provide  more  reliable 
results.  However,  to  date  there  has  been  no  evidence  of  this  adjustment. 

ACTEDS  branch  personnel  are  justified  in  being  hesitant  to  accept  an  expensive 
forecasting  model  that  has  not  proved  to  be  reliable,  and  is  very  difficult  to  use.  As 
mentioned,  the  output  screens  of  the  IFS  report  data  at  a  level  of  detail  which  requires 
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manual  aggregation  over  numerous  paygrades  and  MACOMs.  Unfortunately,  GRC  has 
made  it  clear  that  they  do  not  intend  to  do  other  than  minor  parameter  adjustments  to  the 
IFS.  They  expect  the  U.S.  Army  to  accept  their  model,  even  if  it  is  practically  better  for 
ACTEDS  branch  personnel  to  conduct  the  forecasts  manually. 

In  summary,  there  are  a  number  of  relatively  straight  forward  improvements  GRC 
can  make  to  the  IFS.  As  already  stated,  GRC  has  declared  their  intentions  of  adjusting  the 
condition  parameters  which  the  IFS  uses  to  determine  which  forecasting  model  to  use.  By 
working  more  closely  with  ACTEDS  branch  personnel,  GRC  could  gain  a  better 
understanding  of  the  seasonality  of  the  data,  and  correct  the  seasonal  parameters  in 
Winter's  model.  Additionally,  the  scope  of  the  IFS  should  be  adjusted  to  provide  forecasts  at 
the  level  of  detail  that  ACTEDS  branch  personnel  can  use  more  readily.  Lastly,  GRC  could 
easily  improve  the  screen  displays  with  the  intent  of  providing  the  user  with  a  display  that 
they  can  easily  understand. 

As  a  final  note,  it  is  believed  that  GRC  knows  the  right  problem  to  be  solved,  and 
how  to  solve  it.  They  truly  have  the  talent  and  the  resources  to  do  it.  Unfortunately,  they 
have  not  completed  designing  an  appropriate  solution  simply  because  the  IFS  does  not  yet 
meet  the  needs  of  the  user .  They  have  not  developed  the  IFS  to  perform  well  in  the  user's 
environment  as  demonstrated  during  the  operational  tests.  Lastly,  but  no  less  importantly, 
they  have  not  assured  the  user's  satisfaction.  In  actuality,  they  appear  less  interested  in 
satisfying  the  needs  of  the  user,  and  more  interested  in  getting  ACTEDS  branch  to  accept 
the  IFS  in  its  current  form. 
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APPENDIX  A  (Functional  Description) 


1.  Overview 

This  appendix  provides  a  functional  description  of  the  IFS  in  non-computer  terms. 
A  reader  not  schooled  in  the  terms  of  computer  languages  should  be  able  to  understand  the 
way  the  IFS  operates  from  reading  this  alone. 

The  IFS  has  three  main  components;  Database  Management,  Forecasting,  and 
Screen  Display.  The  components  interact  as  depicted  below: 


IFS  COMPONENTS 


figure  1 


Each  of  these  components  work  together  to  provide  the  end  result,  which  is  a 
forecast  of  the  number  of  interns  required  by  career  program  (CP),  Major  Command 
(MACOM),  and  paygrade  (PG). 

2.  Data  Management 

The  IFS  first  reads  the  data  for  all  the  CP/MACOM/PG  categories  for  the  past  48 
months.  The  data  reveal  the  numbers  of  interns  by  CP/MACOM/PG  category  that  have 
been  assigned,  or  transferred  in  or  out  of  the  army.  The  IFS  reads  the  data  for  each 
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category  from  the  Headquarters  U.S.  Army,  Decision  Support  System  (HQ  DSS)  computer 
located  in  the  basement  of  the  Pentagon.  The  HQ  DSS  data,  which  have  been  gathered 
from  all  the  MACQMs,  often  have  errors  that  require  correction.  The  Data  Management 
component  reads  all  the  data,  and  edits  data  that  obviously  are  in  error.  Once  the  data  have 
been  read  and  corrected,  they  are  sent  to  the  Forecasting  component. 

3.  Forecasting 

The  Forecasting  component  has  two  modules  as  depicted  in  figure  2;  the  Inventory 
Projection  Module  (IPM),  and  the  Accession  Computation  Module  (ACM). 


Forecasting  Modules 


V 

r~ . 

Inventory 

r -  ■  l"-"  ■  ' 

Accession 

Projection 

Computation 

figure  2 

The  IPM  takes  the  provided  data,  and  analyzes  each  category,  searching  for  the 
following  transactions: 


Projected  Accessions 
MACOM  Transfers 
Retirements 
Other  Losses 

Career  Program  Migrations 
Promotions 
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For  each  transaction  type  the  IPM  notes  the  number  of  that  type  of  transaction  that 
has  occurred  in  the  past  48  months.  This  is  then  added  together  into  the  respective 
quarters  that  the  months  make  up.  At  this  point  the  IPM  conducts  two  condition  tests  to 
determine  which  forecasting  model  to  use.  The  condition  tests  are  called  "Small  Cell  Size" 
and  "Sufficient  History".  Again,  it  should  be  noted  that  the  IPM  do  these  tests  for  each 
CP/MACOM/PG  category,  and  for  each  transaction  type  within  each  category. 


CONDITION  TESTS 


figure  3 


The  Small  Cell  Size  condition  checks  each  of  the  16  past  quarters  to  see  if  any  of  the 
quarters  have  less  than  25  (which  GRC  claims  they  have  changed  to  10)  entries.  Next,  the 
Forecasting  component  checks  the  Sufficient  History  condition  by  checking  each  of  the  past 
16  quarters  to  see  if  at  least  12  of  them  have  entries  that  are  greater  than  zero.  If  the  data 
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for  a  particular  CP/MACOM/PG  category  and  transaction  type  passes  both  of  the  condition 
tests,  then  a  forecasted  transaction  rate  is  computed  using  Winter's  Approximation  Model 
(Winter's).  If  either  condition  fails,  then  a  forecasted  transaction  rate  is  computed  using  the 
simple  Weighted  Moving  Average  Model  (WMA). 

Once  the  IPM  has  computed  transaction  rates  for  each  type  of  transaction  by 
CP/MACOM/PG  category,  these  rates  are  then  used  to  compute  the  projected  losses  for  each 
category.  This  is  done  by  multiplying  the  rates,  in  the  sequence  described  above,  by  the 
current  intern  inventory  by  category.  The  resulting  product  is  then  subtracted  (or  added  in 
the  case  of  projected  accessions)  from  the  current  inventory  of  that  category,  and  the  next 
forecasted  transaction  rate  is  multiplied  in.  The  IPM  does  this  process  for  all  categories. 

Once  completed  the  IPM  compares  the  resulting  inventory  to  the  beginning 
inventory  to  determine  the  projected  losses.  These  losses  are  reported  to  the  Accession 
Computation  Module  (ACM).  Lastly,  the  IPM  compares  the  current  intern  inventory  with 
the  required  strength  as  dictated  by  the  Total  Army  Authorization  Document  (TAADS)  and 
reports  the  differences  to  the  ACM. 

The  ACM  computes  the  required  intern  accessions  by  subtracting  the  losses  from  the 
current  inventory,  and  determining  the  number  of  interns  graduating  during  the  current 
period.  Next,  the  ACM  accounts  for  journeymen  positions  to  be  filled  with  interns.  This 
causes  a  loss  of  interns  at  the  grade  of  GS-09  and  GS-11.  GRC  claims  that  their  code  only 
accounts  for  the  locally-funded  and  centrally-funded  interns  to  fill  journeymen 
requirements. 
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Screen  Display 


Finally,  the  ACM  combines  all  the  provided  information  and  provides  the  projected 
lired  intern  accession  to  the  output  screen  for  the  users  to  view.  In  addition,  projected 
es  can  be  displayed  to  the  output  screen.  Since  the  users  of  the  IFS  are  already  familiar 
i  the  screen  display,  no  further  description  is  offered. 

Msclaimer 

In  the  way  of  a  disclaimer,  the  above  description  is  how  the  IFS  works  as  understood 
he  Operations  Research  Center  personnel.  This  description  was  prepared  based  upon  a 
e  reading  of  the  provided  White  Paper,  an  examination  of  the  computer  code  for  the  IFS, 
interviews  with  GRC  personnel.  Additionally,  this  disclaimer  highlights  the  need  for 
software  product  to  have  a  configuration  management  plan.  For  example,  the  version 
?S  evaluated  by  the  ORCEN  will  hopefully  be  followed  by  a  new  and  improved  version  of 
.  There  has  been  some  confusion  as  whether  the  version  evaluated  already  incorporates 
e  changes  to  parameters.  Each  version  of  IFS  needs  to  be  documented  and  labeled  so 
.  the  users  and  developers  can  maintain  current,  accurate  copies. 
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APPENDIX  B  (Statement  of  Work  Discrepancies) 


1.  Overview 

Task  Order  Number  CIV-18A-87,  The  Army  Civilian  Forecasting  system 
(CIVFOKS),  Contract  MDA903-87-C-0860,  paragraph  6.2  states: 

Prepare  Functional  Description/System  Specification  for  the  Intern 
Forecasting  System  (IFS)  in  accordance  with  DoD  Standard  7935. 1-S. 

Design  and  develop  the  modules  described  by  the  system 
specification. 

Test,  modify  and  implement  the  IFS  to  ensure  the  accuracy  of  data 
generated  and  operational  capabilities  of  the  system  to  provide  "what  if’ 
capabilities. 

Provide  all  necessary  operations,  maintenance,  training,  analysis 
and  documentation  support  to  the  IFS  user  community. 


The  following  paragraphs  are  organized  by  the  functions  specified  in  paragraph  2.2, 
System  Functions  of  the  IFS  System  Specification.  They  describe  what  has  been  found  to  be 
either  inadequate  or  unfulfilled.  This  is  only  a  partial  listing.  Other  requirements  within 
the  Systems  Functions  have  not  been  totally  fulfilled  as  well.  Only  the  major  shortcomings 
have  been  stated  here. 
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2*  Strength  Management  Capabilities 


A.  Provide  data  on  the  national  labor  market  to  assess  constraints  on  the  supply 
of  available  labor. 

-Not  included  in  the  delivered  product. 

B.  Develop  projections  by  various  combinations  of  MACOM/IRA,  occupational 
series,  career  program,  grade,  academic  discipline,  gender,  race/national  origin, 
age  and  service  computation  date. 

-Not  all  combinations  available  in  the  delivered  product. 

3.  Analysis  and  Presentation  Capabilities 

A.  Provide  an  interactive,  user-friendly,  quick  response  Decision  Support 
system  (DSS)  equipped  with  a  data  retrieval  capability  to  present  historical  and 
projected  data.  The  system  must  provide  both  tabular  and  graphic  displays 
which  can  be  reproduced  in  hardcopy  form. 

-Graphic  displays  are  not  available  in  delivered  product. 

B.  Support  various  program/functional  areas  within  TAPC-CPS  by  allowing  the 
users  to  interact  with  the  system  at  various  points  by  exercising  input/control 
options  such  as: 

1.  modification  of  separation,  promotion,  and  transfer  rates. 
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2.  specifying  workforce  level  constraints  on  the  career  program  workforce. 

3.  specifying  budget  constraints  on  the  ACTEDS  branch  funding  levels 
available  for  the  intern  program. 

4.  specifying  the  length  of  internships. 

5.  model  "priority  of  fill"  given  a  budget  scenario  and  projected  supply 
constraints. 

-Not  all  options  available  in  the  delivered  product . 

C.  Provide  work  force  profiles  on  an  "as-needed1  basis. 

-Not  available  on  the  output  screen . 

D.  Provide  quarterly,  semi-annual,  and  annual  reports  to  support  analysis  of 
the  intake  and  retention  of  interns  by  source  of  intake,  race/national  origin, 
academic  discipline,  gender,  and  education  level  for  total  Army  and  each 
MACOM/IRA.  This  includes  an  analysis  of  intern  losses  (during  the  internship 
and  after  graduation  for  each  year)  by  loss  reason,  career  program,  race/national 
origin,  academic  discipline,  gender,  and  educational  level  for  total  Army  and 
each  MACOM/IRA  by  geographic  region. 

- Not  all  reports  are  provided  in  the  delivered  product . 


4.  Training  Management  Capabilities 

A.  Provide  reports  on  the  number  of  interns  who  are  past  their  graduation  date 
but  who  are  still  counted  against  HQDA  authorizations. 
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-This  capability  is  not  provided  in  the  delivered  product. 


5.  Conclusion 

One  can  easily  determine  that  the  IFS  did  not  contain  a  number  of  capabilities 
described  in  the  systems  specifications  listed  in  the  Statement  of  Work  .  GRC  has  provided 
reasons  for  the  exclusions  to  ACTEDS  branch.  In  general,  GRC's  reasons  do  not  seem  to 
adequately  explain  their  shortcomings. 
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